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COCHLEAR IMPLANT SOUND PROCESSING METHOD AND SYSTEM 

^ The present invention relates generally to the generation of electrical stimuli 
for application to an auditory prosthesis electrode array forming part of a cochlear 
5 implant. 

Cochlear implants are now the clinical device of choice for people who have 
a profound hearing loss. Such implants typically consist of an internal and an 
extemal component. The internal component consists of an array of electrodes 
implanted within the cochlea, and associated electronics for providing current 

10 stimulation to the surviving auditory neural elements. The extemal device is a 
sound processor, which converts an incoming acoustic signal into appropriate 
electrical stimulation pulses for application to the electrode array. 

Cochlear implant technology has progressed to the stage that the average 
user can expect to understand conversational speech in quiet surroundings without 

15 the aid of lip reading. However, cochlear implants do not provide a satisfactory 
hearing experience when listening to speech in the presence of other noises, and 
when listening to musical sounds and in particular when listening to melody. There 
are thought to be two principle reasons for this: firstly, the pitch of complex 
sounds, generally related to the fundamental frequency, is poorly conveyed by 

20 current speech processing schemes, and secondly, the fine temporal and spectral 
information that normally hearing people use to segregate a complex sound from 
noise or other complex sounds is lost or reduced in implant processing. 

Fine temporal information in acoustic sounds is represented by relatively fast 
amplitude fluctuations in the wave form of the signal. For periodic (or harmonic) 

25 complex signals, such as voiced speech or musical sounds, the fundamental 
frequency (FO) is related to the period of fluctuation, and leads to the perception of 
musical pitch or voice pitch. The fundamental frequency of these sounds generally 
lies between 50 Hz and at least several hundred Hz. Finer temporal information is 
provided by the frequencies of the harmonic components of such sounds, which are 
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at multiples of the fundamental frequency. 

The signal processing applied in current cochlear implant sound processors 
does not allow the fine frequency perception that this would require, either 
spectrally or temporally. In the spectral dimension, up to about twenty fixed narrow 
5 band filters are usually assigned tonotopically to the electrode positions in the 
cochlea. This arrangement does not allow the implantee to resolve harmonic 
components of complex sounds, and hence to perceive the fundamental frequency 
via the spectral template method. In the temporal domain, the output of the filters 
will have amplitude modulations at the fundamental frequency rate, provided that 
10 the filters are wide enough to encompass more that one harmonic, and provided that 
smoothing functions at the output of the filters do not attenuate this modulation. In 
theory, these modulations could provide a fundamental frequency pitch perception, 
provided that the rate of stimulation is high enough to accurately "sample" this 
modulation pattern. 

15 However, recent studies have shown that users of one of the most common 

processing strategies used in cochlear implants, namely the Spectral Peak or 
SPEAK strategy, could not discriminate the pitch of some complex sounds. 
Inspection of the processor output showed amplitude modulations for low 
fundamental frequencies, but the modulations on different electrodes appear to have 

20 somewhat random phase relationships. It has also been shown that phase shifts on 
nearby electrodes will impair the ability of the implantee to perceive the modulation 
frequency, or cause them to perceive a pitch not related to the fundamental 
frequency. 

It would therefore be desirable to provide a cochlear implant sound 
25 processing strategy to improve the perception of fundamental frequency for speech 
and musical stimuli. 

It would also be desirable to p rovide a c ochlear implant s ound p rocessing 
strategy to allow the important components of a complex sound, such as a vowel 
sound or a musical note, to be perceptually grouped by an implantee. 
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It would also be desirable to provide a cochlear implant sound processing 
strategy to allow two or more concurrent complex sounds, such as two speakers, to 
be better perceptually segregated by an implantee. 

It would also be desirable to provide a cochlear implant sound processing 
5 strategy that ameliorates or overcomes one or more disadvantages of the prior art. 

With this in mind, one aspect of the present invention provides an auditory 
prosthesis device for selectively stimulating electrodes within an auditory prosthesis 
electrode array, comprising: 

a transducer for converting a complex acoustic sound into an electrical 

10 signal, 

signal processing means responsive to an electrical signal and generating a 
temporal pattem of stimulation pulses to selected electrodes within the electrode 
array, the stimulation pulses being applied to each electrode at an electrode 
stimulation rate, 

15 feature extraction means for deriving an estimate of at least one fundamental 

frequency of the electrical signal, and 

stimulation pulse adjustment means for adjusting the stimulation pulses in 
accordance with the estimated fundamental frequency. 

In a preferred embodiment of the invention, the feature extraction means 
20 comprises a plurality of fundamental frequency templates separating the electrical 
signal into a plurality of different frequency components, each template comprising 
a first template filter centred on a first frequency and one or more further template 
filters centred on harmonics of that first frequency, wherein 

the stimulation pulse adjustment means acts to compare output signals from 
25 the template filters in each template to determine a matching template passing 
maximum power compared to the remaining templates, the first frequency of the 
matching template being used by the stimulation pulse adjustment means as the 
estimated fundamental frequency. 

The stimulation pulse adjustment means may selectively generate stimulation 
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pulses during an electric stimulation period, the pulse adjustment means acting to 
convert the estimated fundamental frequency into the electric stimulation period 
using a specific conversion function. 

In one embodiment, the specific conversion function may perform a first 
5 function of setting the longest interval between consecutive pulses in the temporal 
pattern during the electric stimulation period in accordance with the estimated 
fundamental frequency. 

Additionally, or altematively, the specific conversion function may perform 
a second function of setting the electric stimulation period in accordance with the 
10 estimated fundamental frequency. 

The specific conversion function may vary between performing the first and 
second function. 

The specific conversion function may preferentially perform the second 
function when the electrodes to be stimulated in the temporal pattern are physically 
15 proximate each other. 

In the presence of two or more complex acoustic sounds, the feature 
extraction means may derive an estimate of multiple fundamental frequencies from 
the electrical signal and the specific conversion function may convert the multiple 
fundamental frequencies into a corresponding number of interleaved electric 
20 stimulation periods. 

In another embodiment of the invention, the stimulation pulse adjustment 
means may modulate the amplitude of the stimulation pulses on all or a subset of 
activated electrodes at a modulation rate given by the estimated fundamental 
frequency. 

25 The stimulation pulse adjustment means may further act to process the output 

signals from the template filters in the matching template to determine an estimate 
of one or more formants of the electrical signal. 

The stimulation pulse adjustment means may modulate the amplitude of the 
stimulation pulses applied to one or more electrodes within the electrode array 
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which correspond to the estimated one or more formants. 

When the feature extraction means derives an estimate of multiple 
fundamental frequencies, each corresponding to a different complex acoustic sound, 
from the electrical signal, the stimulation pulse adjustment means may temporally 
5 segregate the amplitude modulation applied to the stimulation pulses for each 
different complex acoustic sound. 

In a further embodiment, the stimulation pulse adjustment means may act to 
apply a temporal offset to the stimulation pulses applied to those electrodes within 
the electrode array which correspond to the estimated fundamental frequency or 
10 formants o f the electrical s ignal with respect to the stimulation pulses applied to 
other electrodes in the electrode array. 

The stimulation pulses may be applied simultaneously to those electrodes 
within the electrode array which correspond to the harmonic components of the 
complex sound, or formant frequencies of the acoustic signal. 
15 Alternatively, the stimulation pulses applied to those e lectrodes w ithin the 

electrode array which correspond to the harmonic components of the complex 
sound, or formant frequencies of the acoustic signal, may be temporally segregated 
from the stimulation pulses applied to the other electrodes in the electrode array. 

Another aspect of the invention provides a bilateral auditory prosthesis 
20 apparatus including two auditory prosthesis devices as described hereabove. 

The stimulation pulse adjustment means of a first of the auditory prosthesis 
devices may act to determine a matching template passing maximum power 
compared to the remaining templates in the first auditory prosthesis device, the first 
auditory prosthesis device further acting to determine the power passed by the 
25 corresponding template in the second auditory prosthesis device and, if less than the 
power passed by the matching template in the first auditory prosthesis device, 
adjusting the stimulation pulses in the first auditory prosthesis device in accordance 
with the estimated fimdamental frequency. 

In the presence of two complex acoustic sounds, the stimulation pulse 
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adjustment means of the first auditory prosthesis device may act to determine to 
matching templates passing power maxima compared to the remaining templates in 
the first auditory prosthesis device, the stimulation pulse adjustment means of the 
first auditory prosthesis device adjusting the stimulation pulses in the first auditory 
5 prosthesis device in accordance with the estimated fundamental frequency 
corresponding to a first power maximum, and the stimulation pulse adjustment 
means of the second auditory prosthesis device adjusting the stimulation pulses in 
the second auditory prosthesis device in accordance with the estimated fundamental 
frequency corresponding to the second power maximum. 

10 The stimulation pulse adjustment means of the first auditory prosthesis 

device may act to apply stimulation pulses to one or more electrodes in the first 
auditory prosthesis device which correspond to one or more formants of the 
complex acoustic sounds, and the stimulation pulse adjustment means of the second 
auditory prosthesis device may act to apply stimulation pulses, which are de- 

15 correlated with the stimulation pulses applied by the first auditory prosthesis device, 
to one or more electrodes in the second auditory prosthesis device which 
corresponds to the same one or more formants. 

A further aspect of the invention provides a method of operating an auditory 
prosthesis device for selectively stimulating electrodes within an auditory prothesis 

20 electrode array, the method including: 

converting a complex acoustic sound into an electrical signal, 
responsive to the electrical signal, generating a temporal pattem of 
stimulation pulses to selected electrodes within the electrode array, the stimulation 
pulses being applied to each electrode at an electrode stimulation rate, 

25 deriving an estimate of at least one fundamental fi-equency of the electrical 

signal, and 

adjusting the stimulation pulses in accordance with the estimated 
fundamental frequency. 

The following description refers in more detail to the various features of the 
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invention. To facilitate an understanding of the invention, reference is made in the 
description to the accompanying drawings where the auditory prosthesis device and 
method of its operation are illustrated in a preferred embodiment. It is to be 
understood however, that the invention is not limited to the preferred embodiment 
5 as shown in the drawings. 
In the drawings: 

Figure 1 is a schematic diagram showing various components of the sound 
processor of one embodiment of an auditory prosthesis device according to the 
present invention; 

10 Figure 2 is a graphical representation of the pass bands of the first four 

template filters within each of three fundamental firequency templates forming part 
of the sound processor of Figure 1, and a harmonic input spectrum to be processed 
using these templates; 

Figure 3 is a graphic representation of a temporal pattem of electrical 

15 stimulation pulses applied to an electrode array in an auditory prosthesis device; 

Figure 4 is a flow chart showing schematically the series of functional 
operations performed by the sound processor of Figure 1, and 

Figure 5 is a schematic diagram of a part of bilateral auditory prosthesis 
apparatus including two sound processors. 

20 Existing auditory prosthesis devices include a microphone that converts 

sounds into electrical signals, and a sound processor responsive to the electrical 
signal from the microphone and selectively generating stimulation pulses for 
application to electrodes within an electrode array. In multi-channel cochlear 
implants, the electrode array is inserted in the cochlea so that different auditory 

25 nerve fibres can be simulated at different places in the cochlea. Typically, the 
electrode array includes 22 electrodes. Different electrodes in the electrode array 
are stimulated by the pulses from the sound processor depending upon the 
frequency of the electrical signal received from the microphone. The electrodes 
near the base of the cochlea are stimulated with electrical pulses derived from high 
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frequency acoustic information, while electrodes near the apex are stimulated with 
electrical pulses derived from low frequency acoustic information. The sound 
processor acts to break the input signal into different frequency bands and 
selectively apply stimulation pulses to different electrodes within the electrode array 
5 based upon the signal in each of the frequency bands. Typically, the microphone 
and speech processor are wom by the patient, and stimulation pulses from the sound 
processor are transmitted to the electrode array by a transmitter/receiver pair. 

Figure 1 shows a more detailed view of the various components of the sound 
processor. The sound processor 10 implements, in this embodiment, a Continuous 

10 Interleaved Sampling (CIS) sound processing strategy. The electrical signal from 
the microphone 2 is firstly pre-emphasised and then passed through a bank 11 of 
fixed overlapping bandpass filters. In this example, there are 22 bandpass filters 

(BFPl, BFP2 BPF22), corresponding to the 22 electrodes within the electrode 

array 4. The envelopes of the filtered waveforms are then extracted by a bank 12 of 

15 full wave rectifiers and low-pass filters. The envelope outputs are then compressed 
and used by a pulse generator 13 to modulate biphasic pulses applied to the 

electrodes (EH, El-2, El-22) in the electrode array 4. In order to avoid channel 

interaction, trains of the biphasic pulses are delivered to the electrodes in anon- 
overlapping or non-simultaneous fashion, i.e., in a way such that only one electrode 

20 is stimulated at a time. The amplitude of the pulses is related to the amplitude of 
the envelopes detected by the envelope detectors. 

Whilst the sound processing strategy used in this embodiment of the 
invention conforms to the CIS approach, it will be appreciated that other strategies 
that involve the passing of the electrical signal through a bank of fixed overlapping 

25 bandpass filters, the outputs of which determine the amplitude of the pulse applied 
to the corresponding electrode, may also be adopted. Examples of such strategies 
include the Advanced Combination Encoders (ACE) strategy, and the Spectral Peak 
(SPEAK) strategy. Alternatively, the amplitude of the stimulation pulses applied to 
each electrode may be determined from an acoustic loudness model together with 
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an electrical loudness model, such as in the Specific Loudness (SpeL) strategy. 

The sound processor 10 further comprises a bank 14 of fundamental 
frequency templates for deriving an estimate of at least one fundamental frequency 
of the electrical signal from the microphone 2. The bank 14 of fundamental 
5 frequency templates separates the electrical signal from the microphone 2 into 
different frequency components. Each fimdamental frequency template comprises a 
first template filter centred on a first frequency within the frequency component and 
one or more further template filters centred on harmonics of that first frequency. In 
the example depicted in Figure 2, the fundamental frequency template FOi includes 

10 a first template filter centred in practice on a fundamental frequency FOi, and 3 
template filters centred respectively on harmonics of that frequency, namely 2F0i, 
3F0i and 4F0i. In practice though, a fundamental frequency template may include 
any number of filters, and may typically include ten filters. 

Similarly, the fundamental frequency templates FO2, FO3 FO12 shown in 

15 Figure 1 each include a first template filter centred on a fundamental frequency and 
a series of filters centred respectively on harmonics of that frequency. 

The frequency spacing between the fundamental frequency on which the first 
template filter in each template is centred will depend upon the ability of the 
cochlea implantee to distinguish sounds. Typically, the values of the fundamental 

20 frequencies detected by the templates m ay b e derived from the standard musical 
scale. In one embodiment, the fundamental frequencies may be spaced from each 
other by a semi-tone. 

Each fundamental frequency template defines an ordered list of frequencies. 
Each frequency specifies the lower or upper sound of a different fundamental 

25 frequency and its associated harmonics that may potentially be present in the signal 
detected by the microphone 2. The upper and lower sounds may be spaced by a half 
semi-tone on either side of the centre frequency for each harmonic. For example, 
the fundamental frequency template for the note "A" would define the upper and 
lower bounds of the first template filter at 213.7 Hz and 226.4 Hz (220 Hz ± 0.5 

004443486 



10 

semi-tone), and the upper and lower bounds of the subsequent template filters at 
427.5 Hz and 452.9 Hz, 641.2 Hz and 679.3 Hz, 854.9 Hz and 905.8 Hz, and so on. 

The output signals from the template filters within each fundamental 
frequency template filter are provided to a stimulation pulse adjustment device 15. 
5 Figure 2 illustrates the pass bands 20, 21 and 22 of the first four template 

filters in each of the first three fundamental frequency templates FOi, FO2, and FO3 
shown in Figure 1. Input signal power that is passed by the template filters with the 
template FOi for example, is assumed to arise from a harmonic signal with a 
fundamental frequency approximately equal to FOi, whereas the input power that is 

10 not passed by the filters within this template is assumed to arise from noise. 

An example of a harmonic input spectrum 23 of a signal from the 
microphone 2 to be processed by a set of templates including template filters having 
the pass bands shown in Figure 2 is also depicted. The input spectrum has a 
fundamental frequency of FOx. The lowest four harmonics of FOx are shown, 

15 starting at FOx. Although the magnitudes of the harmonic components vary, it can 
be seen from this figure that the largest amount of power contained in the spectrum 
would be passed by the template having filters centred on FO2 and its harmonic 
components, when compared to the amount of power passed by the other templates. 
Accordingly, the fundamental frequency FO2 can be taken to be an estimate of the 

20 value of the fimdamental frequency FOx. 

The pulse adjustment device 15 compares the signals from the filters in each 
of the templates in the bank 14 to determine the template p assing the maximum 
power compared to the other templates. The first frequency upon which the first 
filter within that template filter is centred is taken to be an estimate of the 

25 fundamental frequency of the signal from the microphone 2. 

Furthermore, the output signals of the filters within the template passing the 
maximum power can be processed by the pulse adjustment device 15 to extract 
local spectral peaks. In effect, the filters within that template act to sample the 
complex harmonic signal detected by the microphone 2, and further processing of 
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this sampled signal can provide an estimate of the fomiant frequencies in the case of 
voiced speech or a similar type of sound. 

To better determine the template(s) passing the maximum power compared 
to the other templates, a weighting function may be applied by the pulse adjustment 
5 device 15 to the outputs, of all filters in the templates. For example, a weighting 
may be applied to increase the value of the output signal strength of those templates 
centred on a fundamental frequency in the mid-range of human hearing. For each 
template, the outputs of the filters are amplified or attenuated according to their 
centre frequencies. The range of centre frequencies over which an appropriate 
10 waiting function is applied may typically be between 500 to 1500 Hz. 

It is to be understood that in other embodiments of the invention, other 
techniques can be used to provide a real-time estimate of the fundamental frequency 
of a complex harmonic signal detected by the microphone 2. The choice of an FO 
analysis technique may be made with respect to the nature of the input signal, the 
15 acoustic conditions in which the system will be used and the analysis output errors 
that are acceptable and those that are not. The techniques include time domain 
techniques, frequency domain techniques, hybrid time and frequency domain 
techniques and direct measurement of larynx activity. 

Time domain fO estimation techniques exploit the fact that the waveform of a 
20 periodic sound repeats, and they detect features which occur once per cycle during 
voiced sounds, such as the major positive (or negative) peak, positive- (or negative-) 
going zero crossings, the slope changes associated with major peaks or any readily 
identifiable feature on the waveform which can be identified as occurring once per 
cycle. 

25 Frequency domain fO estimation techniques take advantage of the fact that 

the frequency spectrum of a periodic signal has a series of regularly spaced peaks, 
or 'harmonics', and a non-periodic signal has a continuous spectrum with no 
regularly spaced peaks. FO estimation is typically based on finding and tracking 
either: (a) the fO component (first harmonic) itself; or (b) the frequency difference 
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between any two adjacent harmonics, which is equal to fO; or (c) the fO which best 
'fits' all the harmonics present. 

Hybrid techniques make use of a combination of time and frequency domain 
features such that some merits of one domain might override some drawback of the 
5 other domain. Direct measurement of larynx activity can be based on the use of a 
throat microphone, or electrical impedance measurements systems such as the 
electrolaryngograph or electroglottograph. 

An advantage of the technique for identifying the fundamental frequency of 
the electrical signal described in relation to Figure 1 is that, as stated above, it also 
10 identifies the frequency components of the complex harmonic signal. 

A further advantage of this technique is that it enables the identification of 
two concurrent complex sounds. In this case, there with be maxima at the output of 
two fundamental frequency templates within the bank 14, each maxima 
corresponding to an estimate of a separate fundamental frequency. 
15 Furthermore, the outputs of the filters in these two templates will specify 

which spectral components belong to each fundamental frequency. 

A third advantage is that the signal to noise ratio can be estimated by 
comparing the total power at the output of the matching fundamental frequency 
template with the total power of those components of the signal that do not coincide 
20 with any of the filters in the matching fundamental frequency template. S uch a 
signal to noise ratio estimate is an indication of the reliability of the estimated 
fundamental frequency value(s). 

To that effect, the stimulation pulse adjustment device 15 additionally detects 
the level of the total overall signal captured by the microphone 2 without being 
25 firstly filtered by the bank 14 of templates. The total overall signal is a combination 
of the complex harmonic signals (if any) and noise that is detected by the 
microphone 2. The total overall signal strength may be compared by the pulse 
adjustment device 15 to the signal strength at the outputs of the matching 
fundamental frequency templates to determine either a complex harmonic signal-to- 
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noise ratio or a complex harmonic signal-to-overall signal ratio. If one or both of 
these ratios is below a predetermined threshold value, no pulse adjustment may be 
applied. 

Altematively, no pulse adjustment may be applied if the pulse adjustment 
5 device 15 determines that the overall signal strength is below a predetermined 
threshold value and that the estimate of the fundamental frequency of the complex 
harmonic signal is likely to be unreliable. 

If none of the outputs of the templates are determined by the pulse 
adjustment device 15 are determined to have a power greater than any of the others 
10 (i.e. a high level signal is detected at all templates), no pulse adjustment may also be 
applied. 

Rather than simply applying no pulse adjustment in the above-described 
scenarios, the pulse adjustment device 15 may alternately set the stimulation rate of 
the pulses applied by the pulse generator 13 to either a fixed high rate of stimulation 

15 or a random stimulation rate. 

The estimations of the fundamental frequency (or fundamental frequencies) 
can be used by the pulse adjustment device 15 to modify the "usual" output of the 
pulse generator 13 in a variety of ways. Each of these adjustments allows the main 
components of the complex sound to be perceptually grouped by the cochlear 

20 implantee, and also allows the implantee to better perceive the fundamental 
frequency pitch. 

In a first technique, the stimulation pulse adjustment means applies 
information derived from the estimated fundamental frequency to the temporal 
pattem of pulses generated by the pulse generator 13. Figure 3 shows one example 
25 of such a temporal patter of pulses applied to electrodes within an electrode array of 
a cochlea implant. In this example, pulses are delivered to the electrodes in a non- 
overlapping fashion so that only one electrode is stimulated at a time. Each 
electrode is stimulated at an electrode stimulation period 30 determined by the 
interval between consecutive pulses applied to a same electrode. The temporal 
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pattern of pulses shown in Figure 3 is applied to selected electrodes within an 
electrode array of cochlea implant during an electric stimulation period, or control 
period, referenced 3L The longest interval between consecutive pulses in the 
temporal pattern shown in Figure 3 is referenced 32. 
5 In its simplest form, the stimulation pulse adjustment means can set the 

electrode stimulation rate to be equal to the estimated fundamental frequency. 
However, psycho-physical research has indicated that, where multiple pulses are 
applied to an electrode in an electrode stimulation period, the pitch perceived is 
related to the longest interval in the pattern, rather than the stimulation period. 

10 Furthermore, it has been shown that stimulation on nearby electrodes is perceptually 
amalgamated to form an overall pitch perception. Accordingly, the specific 
conversion function used by the pulse adjustment device 15, in a first embodiment, 
sets the longest interval between consecutive pulses in the temporal pattern during 
the electric stimulation period in accordance with the estimated fundamental 

15 frequency. In this way, an appropriate pitch verses fundamental frequency 
conversion is produced for implantees. The principle is applicable when multiple 
pulses are present in the temporal electrical stimulation pattem for each electric 
stimulation period. 

For example, it is possible to run through each group of active electrodes in a 
20 usual ACE or CIS scheme more than once within each electric stimulation period, at 
least for speech sources or other low values of fundamental frequency. With 
existing, fast cochlea stimulators, eight selected electrodes may typically be 
activated on each of, for example, four stimulation cycles with a time interval that is 
shorter than half the period of a typical speech fundamental frequency. Thus, even 
25 if stimulation gaps of length equal to the period of the fundamental frequency are 
incorporated into the electric stimulation period, an overall high rate of stimulation 
can be achieved. 

The specific conversion function may also be used to set the electric 
stimulation rate at which pulses are applied to individual electrodes in accordance 
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with the estimated fundamental frequency. 

The specific conversion function may also vary between setting the longest 
inter-pulse interval to correspond with the estimated fiindamental frequency, and 
setting the electrode stimulation rate to correspond with the estimated fundamental 
5 frequency. The specific conversion fimction may vary between the two different 
conversions applied to the stimulation pulses in a gradual manner. The specific 
conversion function may preferentially perform the setting of the longest inter-pulse 
interval to correspond to the estimated fundamental frequency p articularly where 
the electrodes to be stimulated in the temporal pattem are physically proximate to 
10 each other. 

Where the bank 14 of templates derives an estimate of multiple fundamental 
frequencies from the electrical signal, thereby indicating the presence of multiple 
complex sounds, the specific conversion function converts the multiple fundamental 
frequencies into a corresponding number of interleaved electric stimulation periods. 
15 It may be that only one or two electrode stimulation cycles are present for each 
electric stimulation period. 

Another technique that may be implemented by use of the pulse adjustment 
device 15 involves the imposition of an in-phase fundamental frequency amplitude 
modulation to the output of the pulse generator 13. In its simplest form, the 
20 amplitude of every activated electrode can be modulated by the pulse adjustment 
device 15 in accordance with the estimated fundamental frequency. 

A more sophisticated strategy may be to modulate the amplitude of the 
stimulation pulses applied to one or more electrodes within the electrode array 
which correspond to the estimated one or more formants. As described previously, 
25 the output signals of the template filters in the matching fundamental frequency 
template can be processed to determine an estimate of one or more formants of the 
complex acoustic sound. 

The concurrent and in-phase amplitude modulation of the stimulation pulses 
applied to the formant electrodes improves perception of the fundamental frequency 
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pitch of the complex sound. Moreover, the "marking" of the formants with an 
amplitude modulation corresponding to the fundamental frequency of the complex 
sound allows these important components to be perceptually grouped as belonging 
together, and segregated from any concurrent sounds or noise. 
5 Keeping the modulation pattem smooth as the fundamental frequency 

changes can be achieved, for example, by using a modulation weighting function in 
which the phase of the function varies smoothly. That is, the phase of the weighting 
for the next pulse can be an increment of the previous phase, the size of the 
increment being determined by the next fundamental frequency estimate and the 
10 time elapsed. 

The above-described techniques can be used in the case of two or more 
concurrent complex sounds. Where the bank 14 of templates derives an estimate of 
two fundamental frequencies, each corresponding to a different complex acoustic 
sound, there will be two templates that provide significant output signals, or power 

15 maxima. In this case, there will be two fundamental frequencies each with different 
formant estimates provided by the output of two matching templates within the 
bank 14 of fundamental frequency templates. Thus, there may be four electrodes 
which correspond to the estimated formants, which are marked with appropriate 
amplitude modulation patterns. The different modulation rates on the two different 

20 pairs of electrodes will enable the formants to be paired to make correct vowels and 
for the two vowels, or musical sounds, to be heard as perceptually separate. 

An alternative arrangement involves the stimulation pulse adjustment device 
applying a temporal segregation of the amplitude modulation applied to the 
stimulation pulses for each different complex acoustic sound. For example, a 

25 temporal offset may be applied to the stimulation pulses applied to those electrodes 
within the electrode array which correspond to the harmonic components or formant 
frequencies of the acoustic signal, with respect to the simulation pulses applied to 
other electrodes in the electrode array. The stimulation pulses may be applied 
simultaneously to those electrodes within the electrode array which correspond to 
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the harmonic components or formant frequencies of the acoustic signal. 

Altematively, the stimulation pulses applied to those electrodes within the 
electrode array which correspond to the harmonic components or formant 
frequencies of the acoustic signal may be temporally segregated from the 
5 stimulation pulses applied to other electrodes in the electrode array. For example, 
the group of electrodes corresponding to the harmonic components or formant 
frequencies of the acoustic signal may be activated in close succession, following 
by a temporal gap before activation of any additional electrodes. 

As seen in Figure 4, each of the above described variants all have a common 

10 initial processing step in which an acoustic signal 40 is processed in a conventional 
manner, in a first step 41, to determine an electrical level to be applied to each of 
the electrodes in the electrode array 4. In concurrent step 42, the fimdamental 
frequency of the acoustic signal 40 is extracted. Preferably, the formants of the 
complex harmonics signal are also extracted together with the harmonic 

15 components of the signal. At step 43, the information extracted at step 32 is used to 
adjust the simulation pulses conventionally produced by existing sound processing 
techniques. 

The above described techniques can also be adapted to an bilateral auditory 
prosthesis apparatus in which a separate auditory prosthesis device is provided for 

20 each ear of a user. Figure 5 illustrates one example of such a bilateral apparatus. A 
first microphone 50 converts acoustic signals into an electrical signal provided to 
the input of a bank 51 of bandpass filters and envelope detectors, and also to a bank 
52 of spectral template filters. A pulse generator 53 selectively generates 
stimulation pulses to electrodes within a first electrode array implanted in a first ear 

25 of a user. Information relating to the fimdamental frequency of the electrical signal 
is derived from the output of the filters in the bank 52 by a pulse adjustment device 
54. The pulse adjustment device 54 acts to modify the stimulation pulses generated 
by the pulse generator 53 in the manner described in relation to Figure 3. 

Similarly, a second microphone 60 is provided to convert an acoustic signal 
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into an electrical signal provided to the input of a bank 61 of bandpass filters and 
envelope detectors and to a bank 62 of spectral template filters. A pulse generator 
63 acts to selectively generate stimulation pulses to electrodes within a second 
electrode array implanted in the other each of a user. Information relating to the 
5 fundamental frequency of the electric signal, provided by the bank 62 of template 
filters, is used by a pulse adjustment device 64 to modify the stimulation pulses 
generated by the pulse generator 63. In its simplest form, the auditory prosthesis 
devices operate independently and separately process complex sounds detected by 
their respective microphones. However, in the example shown in Figure 5, 

10 processing carried out in one auditory prosthesis device impacts the processing 
carried out in the other auditory prosthesis device. 

The above described information relating to the fundamental frequency of 
the complex signal detected by the microphones 50 and 60 determined by the pulse 
adjustment devices 54 and 64 in Figure 5, are shared between these two devices by 

15 a data bus. 

An important feature of any bilateral fitting is that the frequency to electrode 
allocation in each ear should be arranged so that particular frequency bands are 
allocated to pitch matched electrodes in each ear. In the example shown in Figure 
4, each of the microphones 50 and 60 are arranged on opposite sides of the head, 

20 and separate processing of the signals for each microphone is provided. In this 
case, the pulse adjustment devices 54 and 64 each act to determine a matching 
template passing the maximum power compared to the remaining templates. Each 
of the pulse adjustment devices 54 and 64 then determines the power passed by the 
corresponding template in the other auditory prosthesis device. 

25 If the power passed by this corresponding template is less than the power 

passed by the matching template with which each pulse adjusting device is 
associated, the stimulation pulse adjustment device will adjust the simulation pulses 
as described hereabove. In this way, modification of the stimulation pulses need 
only occur on the side determined to have the larger output signal, this being the 
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side where the dominant sound source is located. The template passing the 
maximum power in each auditory prosthesis device is used to "mark" the electrodes 
corresponding to the features on that side (i.e. with fundamental frequency 
modulations or temporal offsets) or alternatively to create fundamental frequency 
5 based control intervals, provided the power passed by the corresponding template in 
the other auditory prosthesis device is less than the power passed by that matching 
template. 

In the case of two concurrent complex sounds being present, if they are 
spatially separated, separate modifications to the stimulation pulses on each side can 

10 be made. For example, when in the presence of two complex acoustic sounds, the 
pulse adjustment device 54 determines that there are two matching templates 
passing power maxima compared to the remaining templates in the bank 52 of 
templates, the pulse adjustment device 54 can adjust the stimulation pulses 
generated by the pulse generator 53 in accordance with the estimated fundamental 

15 frequency corresponding to the first power maximum, whilst the pulse adjustment 
device 64 adjusts the stimulation pulses generated by the pulse generator 63 in 
accordance with the estimated fundamental frequency corresponding to the second 
power maximvmi. 

In addition, or altematively to the above bilateral scheme, stimulation pulses 
20 applied to electrodes corresponding to one or more formants of a complex acoustic 
sound by one of the auditory prosthesis devices can be de-correlated with 
stimulation pulses applied by the other of the auditory prosthesis devices to the 
corresponding electrodes in that other device. In other words, when the pulse 
adjustment device 54 in the example shown in Figure 5, adjusts pulses applied to 
25 one or more electrodes which correspond to one or more formants of a complex 
acoustic sound, the pulse adjustment device 64 acts to apply stimulation pulses that 
are de-correlated with the stimulation pulses adjusted by the pulse adjustment 
device 54, to one or more electrodes in the other auditory prosthesis device which 
correspond to the same one or more formants. The de-correlation may be effected 
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by applying a different electrode stimulation rate to electrodes corresponding to the 
same formants in each auditory prosthesis device. This can be achieved in practice 
by incrementing the position of these electrodes in the order of electrode activation 
in the temporal pattern of pulses applied to the electrode array each electric 
5 stimulation period. 

It is to be understood that various modifications and/or additions may be 
made to the above described auditory prosthesis device and method for operating 
the auditory prosthesis device without departing jfrom the spirit or ambit of the 
present invention. 
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